Low probability transitions and boundary crossing into disallowed states for a more optimal solution

ABSTRACT

Artificial intelligence (AI) techniques that map disallowed states and enable access to those states under certain conditions through a search algorithm are disclosed. In other words, scenario boundaries may be crossed by jumping from one scenario that is less desirable or even has no solution to another scenario that is more desirable.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Nonprovisional patentapplication Ser. No. 15/968,203 filed May 1, 2018, which is acontinuation-in-part (CIP) of U.S. Nonprovisional patent applicationSer. No. 15/798,020 filed Oct. 30, 2017. The subject matter of theseearlier filed applications is hereby incorporated by reference in itsentirety.

FIELD

The present invention generally pertains to artificial intelligence(AI), and more specifically, to AI techniques that map disallowed statesand enable access to those states under certain conditions through asearch algorithm.

BACKGROUND

Current AI technologies are rule-based systems that utilize Bayesiannetworks or multi-objective optimization techniques to completely mapout the possible decision space and provide the best answer. Forinstance, AI software that has been trained to play chess maps out allof the possible allowed moves and attempts to select the optimal nextmove based on the current sate of the pieces in the game. However, thesetechniques do not map disallowed states. This strict adherence to rulesmakes current AI approaches suboptimal, or even dangerous, for certainapplications. Accordingly, an improved approach may be beneficial.

SUMMARY

Certain embodiments of the present invention may be implemented andprovide solutions to the problems and needs in the art that have not yetbeen fully solved by conventional AI technologies. For example, someembodiments of the present invention pertain to AI techniques that mapdisallowed states and enable access to those states under certainconditions through a search algorithm. In some embodiments, thisalgorithm is robust against limited information scenarios because itassumes all states are possible.

In an embodiment, a computer program is embodied on a non-transitorycomputer-readable medium. The program is configured to cause at leastone processor to map trade space versus time for a plurality ofscenarios, whether reachable or not within rules of a system. Eachpathway in the plurality of scenarios includes a plurality of nodes. Theprogram is also configured to cause the at least one processor to, whilefollowing a pathway from an initial starting point, search for nearbynodes from otherwise unreachable outcomes that provide a bettersolution. When a node is found that is proximate within thepredetermined distance and is associated with a pathway that provides abetter solution, the program is further configured to cause the at leastone processor to jump to the proximate node and follow the path to thebetter solution and output the path with the better solution.

In another embodiment, a computer-implemented method includes performingprincipal component analysis (PCA) on raw data, by a computing system,to generate relational sequences in trade space. The trade spacerepresents relative relationships in the raw data. Thecomputer-implemented method also includes mapping a solution space basedon the generated relational sequences and replaying a plurality ofscenarios, by the computing system, to determine pinch points where pathstructures are proximate to one another and more favorable outcomes canbe jumped to. The computer-implemented method further includesgenerating a new solution path, by the computing system, based on one ormore jumps to more favorable outcomes.

In yet another embodiment, a computer-implemented method includesreviewing all nodes in a pattern, by a computing system, to determine amost likely node for a jump and evaluating whether the jump to the mostlikely node is desirable using a sentiment difference between the mostlikely node for the jump and at least one node that is not previouslyknown to be a valid state, but is still proximate to the most likelynode for the jump. The computer-implemented method also includes, fromall predicted moves, choosing a largest sentiment difference andevaluating all jumpable nodes within a predetermined proximity that arenot predicted by pattern recall, by the computing system. Thecomputer-implemented method further includes selecting a jumpable nodewithin the predetermined proximity with a largest positive sentimentdifference between itself and the most likely node for the jump andoutputting a path with the jumpable node as a path with a bettersolution, by the computing system.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the advantages of certain embodiments of the inventionwill be readily understood, a more particular description of theinvention briefly described above will be rendered by reference tospecific embodiments that are illustrated in the appended drawings.While it should be understood that these drawings depict only typicalembodiments of the invention and are not therefore to be considered tobe limiting of its scope, the invention will be described and explainedwith additional specificity and detail through the use of theaccompanying drawings, in which:

FIG. 1 illustrates a progression from learning to “dreaming” and “luciddreaming,” according to an embodiment of the present invention.

FIG. 2A illustrates UIDs for four image frames, according to anembodiment of the present invention.

FIG. 2B shows a principal component path of WLT_1 vs. WLT_2 for frames0-8 in PCA space, according to an embodiment of the present invention.

FIG. 3A is a graph illustrating success and failure boundaries,according to an embodiment of the present invention.

FIG. 3B illustrates an enlarged portion of the graph of FIG. 3A,according to an embodiment of the present invention.

FIG. 4 is a flowchart illustrating a process for mapping disallowedstates and enabling access to those states under certain conditionsthrough a search algorithm, according to an embodiment of the presentinvention.

FIG. 5 illustrates an example from a computer simulation of a modifiedGalton box game, according to an embodiment of the present invention.

FIG. 6 illustrates a modified software-implemented Galton box, accordingto an embodiment of the present invention.

FIG. 7 illustrates another modified software-implemented Galton box,according to an embodiment of the present invention.

FIG. 8 illustrates yet another modified software-implemented Galton box,according to an embodiment of the present invention.

FIG. 9 illustrates a computing system configured to map disallowedstates and enable access to those states under certain conditionsthrough a search algorithm, according to an embodiment of the presentinvention.

FIG. 10 is a flowchart illustrating a process for providing a path witha better solution, according to an embodiment of the present invention.

FIG. 11 is a flowchart illustrating a process for generating a new, morefavorable solution path, according to an embodiment of the presentinvention.

FIG. 12 is a flowchart illustrating a process for providing a path witha better solution, according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Some embodiments of the present invention pertain to AI techniques thatmap disallowed states and enable access to those states under certainconditions through a search algorithm. In other words, some embodimentsare able to cross scenario boundaries by jumping from one scenario thatis less desirable or even has no solution to another scenario that ismore desirable. A practical application of some embodiments is inautonomous driving situations. For instance, if a car is on a ridesharelane, the law does not allow crossing over the double yellow lines. Ifan exit is approaching, conventional AI would follow the rules ratherthan breaking the law to safely transition to the exit. On the otherhand, human drivers will typically cross the double yellow lines andtake the exit, and their sense of urgency to do so grows as the exitbecomes ever closer.

Another scenario where conventional AI may fail to make the optimalchoice is when a sudden car crash occurs ahead of a vehicle. Aconventional rules-based AI system would cause the vehicle to come to animmediate stop. However, this stop may not be the safest transition(e.g., due to the deceleration forces involved, the risk of anothervehicle colliding from behind, etc.). Accordingly, understanding whatrules can be bent and what rules can be broken for the most beneficialoutcome is important for various applications.

Accordingly, some embodiments map all possible outcomes, including atleast some of those that conventional AI techniques would not consider.Such embodiments may look for the most beneficial outcome regardless ofprobability, and then map the solution space to find the points ofclosest transition between the initial starting point and the desiredendpoint. Some embodiments may jump between these states in order toeffect a result that cannot be realized with conventionalmulti-objective optimization techniques. This also enables the AI systemof some embodiments to act in a reasonable way with limited information,which is more natural in real scenarios that require split-seconddecisions.

Another application of some embodiments is in trade negotiations. Whentwo parties are deadlocked with no desire/ability to reach a deal, theapplication of some embodiments maps out all possible no-win scenarios,impossible win-win scenarios, and bridges these two states at the pointof least contention (i.e., the path of least resistance). Bridging thesetwo states enables the parties to transition between the no-winscenarios and the win-win scenarios.

FIG. 1 illustrates a progression 100 from learning to “dreaming” and“lucid dreaming,” according to an embodiment of the present invention.The system is first trained how to make classifications. On that basis,the system performs a “dreaming” step that starts with a classificationand yields a most probable outcome. Allows the system to engage in“lucid dreaming,” where more desirable possibilities, and the paths toget there, are revealed and implemented.

Dream boundary crossing is an inherent aspect of improvisation. Currentoptimization techniques focus on finding pathways that naturally lead tooptimized scenarios based on the rules of the system. However, therecould be alternative scenarios that can be accessed if at least somepreviously disallowed states are potentially accessible. Theseboundaries are defined herein as “low probability scenarios.” No-winsituations (e.g., the Kobayashi Maru test in Star Trek®) show branchingalternatives that lead to inevitable failure. However, winning scenariosare mapped that have no valid initial starting conditions (e.g., a“cheating” scenario such as that employed by Captain Kirk), there willbe points in which opportunities arise that may enable a failure path tojump into winning scenarios. To access these scenarios, the rules of the“game” must be altered such that the impossible outcome becomespossible.

The AI of some embodiments considers otherwise “impossible” scenarioswith the aim of finding a more beneficial outcome, and then maps thejumps needed to accomplish improvement or success from a failurescenario. Such embodiments enable AI to cross these boundaries as partof its regular programming sequences. This may enable applications asimprovisation during music-making (neutral tasks), as well asidentifying points of low probability that are “near” nodes that canlead to success (e.g., aversion to the failure path, attraction tosuccess paths, etc.).

An algorithm described with respect to some embodiments in parent U.S.Nonprovisional patent application Ser. No. 15/798,020 was created withneutral sentiment suitable for simulation. This algorithm has beenmodified to enable positive and negative sentiments suitable forintelligent navigation of the decision space. The algorithm of someembodiments was further modified to alter its nearest neighbor tradespace search with time, enabling it to cross dream boundaries at weakpivot points.

As with parent U.S. Nonprovisional patent application Ser. No.15/798,020, some embodiments employ multiple unique IDs (UIDs) to mapand look for adjacent, more desirable patterns. UIDs may be six numbersbetween 1-100 that have nearly one trillion possible combinations. Assuch, the UID sequence will be unique for the data. While embodimentsmay be applied to non-image applications (e.g., writing music), inimage-based embodiments, the six-number UID may be derived from thefirst three principal components of a two-dimensional (2D) wavelettransform of the data (WVLT1, WVLT2, WVLT3) and the first threeprincipal components of the DCT of the data (DCT1, DCT2, DCT3).

An example in the context of mapping weather is shown in FIGS. 2A and2B. A sequence of UIDS 200 is produced for a sequence of satelliteimages. Four UIDS for frames 1-4 are shown by way of example in FIG. 2A.The UIDS for eleven frames in sequence are shown in Table 1 below.

TABLE 1 IMAGE SEQUENCE UIDS FROM PCA Frame: WLT_1 WLT_2 WLT_3 DCT_1DCT_2 DCT_3 0 52 37 53 56 40 60 1 55 38 52 58 40 57 2 58 31 63 63 40 613 67 34 55 70 44 59 4 73 38 52 74 48 58 5 79 58 49 75 68 46 6 74 65 4369 71 40 7 61 71 40 57 72 38 8 59 69 46 54 71 46 9 39 57 45 35 49 43 1032 52 47 30 45 43

FIG. 2B shows a principal component path 210 of WLT_1 vs. WLT_2 forframes 0-8 in PCA space, according to an embodiment of the presentinvention. All six variables cannot be plotted in a 2D, so WLT_1 andWLT_2 are used for illustration purposes. WLT_1, WLT_2, and WLT_3represent the three principal orthogonal axes of the wavelet transform.The PCA algorithm derives these orthogonal axes through analyses of thechanges from a sampling of the dataset. DCT_1, DCT_2, and DCT_3represent the three principal orthogonal axes of the discrete cosinetransform (DCT).

Consolidation of PCA data sequences enables “pinch points” in PCA spacethat enable different paths to be taken. In this implementation, PCAspace includes the six-dimensional representation of the datasetcomprising of the three principal components of the wavelet transform,which are most affected by positional changes in the images, and thethree principal components of the DCT, which, like the Fouriertransform, are most affected by changes in shading and frequency.However, it should be noted that other image dimensions, and otherrepresentations for non-image data, may be employed without deviatingfrom the scope of the invention. The key of some embodiments is toemploy a mechanism for representing data in a manner that has a highdegree of uniqueness.

FIG. 3 is a graph 300 illustrating success and failure boundaries,according to an embodiment of the present invention. As can be seen, afailure scenario 310 and a success scenario 320 are mapped in tradespace, each made up of decision points and paths between them that canbe taken over time. However, an initial starting point 312 is located onfailure path 310. Using conventional AI techniques, starting point 312would always lead to failure over time.

Trade space is an arbitrary representation of an underlying activity,such as trade negotiations, lanes of a freeway and correspondingactions, mapping the differences between the spot prices of gold vs. therelative cost of a mining employee and his or her associated overheadand equipment needs, etc. The relative mapping of these differences canbe represented by a direct translation of the image in some embodiments,as in the case of an overhead freeway image, but can also be representedby alternate visualizations including, but not limited to, the plot ofthe relative cost of a mining employee vs. the spot price of gold. Inone reduction to practice, a relatively simple pachinko machine sequencewas used to illustrate the concept.

However, in some embodiments, different paths that do not includestarting point 312 are also mapped, such as success path 320 on theright. Per the above, such embodiments look for pinch points betweenfailure paths and success paths to determine whether a point on afailure path and a point on a success path are proximate to one anotherin PCA space within a certain predetermined amount. In the case of aPachinko machine, the relative possible location of the steel ball mayprovide an indicator of probability—the closer the ball is to anotherstate, the more likely it is that the other state is possible.

As seen more clearly in FIG. 3B, two decision points in pinch point 330are proximate to one another in PCA space—one from failure path 310 andthe other from success path 320. The application may “jump” across fromfailure path 310 to success path 320 at pinch point 330. The applicationmay be encouraged to do so by programming a bias into the logic toprefer success path decision points in pinch points. This bias may growover time, decreasing the proximity in PCA space that is required for apinch point. For instance, the bias may start as neutral, or even asnegative towards the success path, but then shift over time toincreasingly favor the success path.

Procedure to Map, Find, and Solve Decision Space

In some embodiments, trade space versus time is mapped for allscenarios, whether possible or not within the rules of the system. Thismay involve random seeding of time and space, and mapping of scenariosensuing from the map. Scenarios with cohesive paths to success (e.g., aGalton box, such as in Plinko™) are more desirable, and the AI may beinfluenced towards these more desirable outcomes. In some embodiments,multiple success paths may be reachable, but the system may jump to themost desirable success path, or even jump from one success path to amore desirable one. The AI of some embodiments may be driven to nearbysuccess nodes that are not on the current path through random search.

A nonlimiting example of a random search algorithm is for the AI toconsider the next logical step in a previously experienced pattern andcalculate the relative increase or decrease in its desirabilityfunction. Given that step, the AI would then search nearby nodes thathave been experienced but have not previously shown a viable pathway.Those nodes are then evaluated by their desirability, and if a node issufficiently high enough and close enough based on predeterminedthresholds, the AI would then assume that the step is feasible andchoose that step instead. However, any suitable random search techniquemay be used without deviating from the scope of the invention.

As seen in FIGS. 3A and 3B, mapping trade space versus time enables a 2Drepresentation thereof. Using six number UIDs of 1-100 per node shouldbe able to account for a sufficient number of possibilities (indeed,nearly one trillion). Utilizing PCA normalizes components to identifyproximate pinch points and enable the AI to jump from one scenario toanother. Use of desirable, undesirable, and neutral pathways facilitatesbehaviors. Desirability may be a value in some embodiments, and eachnode may be gradient-rated with its own desirability. Undesirability maybe a value and work in a similar fashion, where nodes start as lessnegative and become more negative over time along the path. Comparingthis to the carpool lane problem discussed above, given an alternatepath towards the exit that increases in positive sentiment, thedifference between the increasingly negative path and the increasinglypositive path creates an increasing desire to “cheat” as the driverapproaches the exit. In certain embodiments, neutral conditions mayrepresent unalterable phenomena that cannot be change regardless of whatthe AI decides (e.g., a concrete barrier).

FIG. 4 is a flowchart 400 illustrating a process for mapping disallowedstates and enabling access to those states under certain conditionsthrough a search algorithm, according to an embodiment of the presentinvention. The process begins with performing PCA on the raw data togenerate relational sequences at 410. Trade space is the underlyingspace in which the relative relationships are represented. Morespecifically, trade space is the PCA instantiation of the raw data,which is the underlying dataset. PCA is a class of algorithms thatreduce the data to a representation that may include the most essentialdifferences. Trade space relationships can be as literal as an imagerepresentation to a remapping of the space using alternativevisualization techniques. Alternative visualization techniques could berelatively straightforward, like an edge detection algorithm, or morecomplex, like a support vector machine (SVM) or a full deep learningclassification system. However, any suitable visualization technique maybe used without deviating from the scope of the invention.

The scenarios are then seeded with initial conditions at 420. Theinitial conditions may be seeded with sampled data from the real world,which reveals various alternative paths. If the algorithm has access toa simulator, random states may be initialized to explore more preferablesolutions. The solution space is then mapped at 430. In the case of aGalton box, the x-axis and y-axis represent space, and time is the sumof the sequences of the ball falling down the box.

In this embodiment, the trade space is the PCA representation of theentirety of states available to the system, which is the 6 UID numbersfrom 1-100, giving a possible ˜9×10¹¹ states. Sampling experiences(e.g., data feeds from a car with a driver) show the paths between eachstate. The solution is the navigation of that path to the most favorablesolution. Simulation (somewhat analogous to imagination in human minds)is the mapping of the trade space in areas that have not yet beenexperienced. By using a simulator and initializing with random states,the experiences can be mapped.

A relatively simple example is as follows. A car is driven and data isgathered on what to do on a five-lane highway. The car knows what to doin this situation. The car is then simulated with a random initialstarting point (e.g., the car is suddenly pointed backwards and is goingagainst traffic). The simulator shows that this is a highly negativestate, and through multiple runs, the AI eventually learns that the bestanswer is to immediately go in reverse and perform a “J” turn. The AInow knows what to do then it is driving on a freeway and gets clipped bya lane changing car, for example.

The relational sequences are then saved into positive, negative, andneutral bins at 440. More specifically, solutions that lead to positiveoutcomes are saved in the positive bins, solutions that lead to negativeoutcomes are saved in the negative bin, and solutions that lead toneutral outcomes are saved in the neutral bin. For instance, the neutraloutcomes may be the default for observations of phenomena in order tosimulate natural phenomena that cannot be changed.

The scenarios are then replayed to find pinch points where the pathstructures are “weak” (i.e., proximate to one another) and can be jumpedto more favorable outcomes at 450. This enables the AI to crossrule-based boundaries and jump into new scenarios. The AI then generatesa new solution path at 460 based on one or more jumps to more favorablescenarios.

Per the above, unalterable phenomena may be simulated as neutral paths.The number of times a node has been “seen” in memory may be used toincrease the probability of choosing paths with that node. By relying onreal-life sampling, an indication of probability can be derived. Forexample, if a certain basketball player is observed shooting athree-point shot 100 times, and makes the shot 23 times, one can roughlyestimate that the probability of that player making a three-point shotis 23%. In essence, this sampling creates a preference towards higherprobability nodes, enabling simulation of possible paths with relativeprobabilities.

In some embodiments, the AI is designed to enable success in scenarioswhere the probability of reaching a desired outcome is either low orimpossible using conventional AI rules. By sorting scenarios intopositive and negative bins, variable and desirable outcomes can becreated. Negative decision points may be avoided, while positivedecision points are encouraged. Also, when a negative node isinevitable, the chance that the AI tries to jump between states mayincrease. The decision from the algorithm of some embodiments is basedon both probability and proximity. If the difference between a statethat has a negative sentiment and a state that has a positive sentimentis large enough, and the nodes are close in proximity, then theprobability that the AI will jump that state will typically be high.Probability may be linear, square, cubed, etc., depending on theimplementation. “Jumps” that traverse a larger distance may occur withhigh positive sentiment differences.

One way to incorporate “sentiment” and occurrence frequency into nodesis to include four additional numbers into the UID. For instance,whereas the six-number UlDs discussed above use six numbers from PCA, insome embodiments, two additional numbers are used to represent sentimentand two more additional numbers are used to facilitate patternlock—i.e., how likely the AI is to lock into a pattern including thegiven node. With respect to sentiment, in this embodiment, eachsentiment number encodes a value from 1-100. Two numbers, with amultiplier on the second number, enable 9999 possible values to be used.More numbers, which are used in some embodiments, would provide morenuanced values to choose from.

The pattern lock is utilized in some embodiments in the case that thestates are identical. This occurs, for example, in a game of rock paperscissors, where there are three states, but the sequences of rock paperscissors are what encodes the pattern of interest. The pattern lockenables repeating states to occur and still retain the sequence.

In some embodiments, the trade space is PCA imaging data that inherentlyrepresents areas that are similar as being close to one another. Thesentiment numbers in the UID may also be from 1-100. To figure out thenext step in the path, some embodiments use the difference between nodesof the six trade space numbers plus the two sentiment numbers multipliedtogether. In other words, if the six trade space numbers for a node from1-100 are denoted A, B, C, D, E, F, and the two sentiment numbers from1-100 are denoted G, H, then the difference between a current node X anda next node Y in the path may be found by Eq. (1) below:

((A _(X) +B _(X) +C _(X) +D _(X) +E _(X) +F _(X))+(G _(X) ×H _(X))) −((A_(Y) +B _(Y) +C _(Y) +D _(Y) +E _(Y) +F _(Y))+(G _(Y) ×H _(Y)))  (1)

The sentiment numbers in this embodiment are multiplied by a factor thatenables a balance between the spatial numbers describing trade spaceproximity and the values describing sentiment. There is also a distancefactor that indicates how far of a jump will be performed. The distancebetween UID states represents the relative distance in the underlyingraw data, and represents the degree of plausibility that these statescan be linked with a path. However, different equations may be usedwithout deviating from the scope of the invention, depending on theimplementation. It should be noted that any range of numbers, and anycollective number thereof, may be used without deviating from the scopeof the invention. Furthermore, different ranges may be used for tradespace versus sentiment, etc.

FIG. 5 illustrates an example 500 from a computer simulation of a Galtonbox game, according to an embodiment of the present invention. In thisexample, sentiment is represented by two numbers. In order to achievehigher sentiment values, the second sentiment number is multiplied by arandom value between 0 and 100 such that the first sentiment numberrepresents 1-99 and the second sentiment number represents 0-9900 inthis embodiment. Binary decision points 510 (i.e., left or right) and abarrier 520 are included. There are five “steps” in this embodiment,made up of horizontal rows of decision points 510. If the game were tooperate normally, the “ball” would always end up on the left side ofbarrier 520.

The beginning state is the state of the Galton box, which is when oneputs in a steel ball at the top of the box. As the ball falls, it isattracted to the right-hand side by sentiment. In a real scenario, thiscould be due to magnetism, the tilt of the floor, gravity, etc. The ballencounters barrier 520, and if that state was unallowable, the ballwould progress leftward. However, the algorithm can jump the barrier ifthe sentiment difference is sufficiently large enough, enabling the ballto “jump” the barrier and to move towards the more desirable state.

FIGS. 6-8 illustrate software-implemented Galton boxes 600, 700, 800, aswell as the outcomes that ended up in each “bin”. In Galton box 600 ofFIG. 6, no barriers are present, but a high sentiment area representedby a triangle is provided. This causes simulated balls that approach thehigh sentiment area to tend to want to stay therein. Simulations thatended up in each of the eleven end bins defined by the “pegs” of row 0are shown below Galton box 600. Similarly, simulations that ended up ineach bin with an uncrossable wall (Galton box 700) and a crossable wall(Galton box 800) are shown in FIGS. 7 and 8, respectively. Table 2 belowpresents the results together for comparison.

TABLE 2 SIMULATIONS IN EACH BIN FOR GALTON BOXES IN FIGS. 6-8 GB Num.FIG. Sims 1 2 3 4 5 6 7 8 9 10 11 6 454 1 4 23 27 0 75 130 107 62 25 0 7181 0 2 179 0 0 0 0 0 0 0 0 8 207 1 3 108 0 0 0 12 39 31 3 0

A rock-paper-scissors (RPS) algorithm implementing an embodiment of thepresent invention also consistently came close to matching games withthe top RPS algorithms from http://www.rpscontest.com/. The topalgorithms in RPS contest would learn such that they beat human playersover 80% of the time. The results of this embodiment versus some of thetop algorithms over 200 games are shown in Table 3 below.

TABLE 3 RPS ALGORITHM GAME RESULTS Opponent RPS Algorithm: Wins: Losses:Ties: Dllu1_Randomer 86 90 24 Zai_all_mix_meta 84 91 25 RPS_FA_Fix 86 9321

The RPS algorithm was implemented utilizing the same core logic to mapout pattern space, predict the next move, and in this case, plot a moredesirable outcome by applying the countermove to win the round. This isan example of a dynamically changing system in which the actions of thealgorithm directly affect the future patterns of the opponent. In thiscase, the desirable path was limited to the round. However, the sameGalton box code can be augmented to enable states in which the algorithmdeliberately loses a round or two in order to ensure that later roundsare won more frequently. These “loss leader” strategies are enabled inthe algorithm of some embodiments.

It should be noted that applications of embodiments of the presentinvention are not limited to self-driving cars, image recognition, tradenegotiations, and music. Indeed, embodiments of the present inventionmay be applied for applications including, but not limited to,classification, simulation, and/or decision making.

Example Algorithm

An example algorithm of some embodiments is presented below.

-   -   (1) Look at all nodes in pattern and determine most likely node        jump:

Evaluate the sentiment_difference to understand whether the predictedmove is desirable using Sentiment_difference=(100*G _(x) +H _(x))−(100*G_(y) +H _(y));

-   -   (2) From all predicted moves, choose the best (i.e., largest)        sentiment difference. Note: If there is no sentiment, then it is        a pure simulation and the predicted move is a random draw of the        possible moves recalled.    -   (3) Then evaluate all jumpable nodes that are not predicted by        pattern recall. It is assumed that all nodes can be accessed. It        is either the case that a node has not been previously accessed        or that a barrier exists between two nodes. It is not        necessarily impossible to reach “unreachable” nodes. Rather, it        is another way of saying that just because it has not been done        does not mean it is impossible. This is done by:        -   For all nodes proximate to current node within a            predetermined proximity:

UID_Distance=Sqrt((A _(x) −A _(y))²+(B _(x) −B _(y))²+(C _(x) −C_(h))²+(D _(x) −D _(y))²+(E _(x) −E _(y))²+(F _(x) −F _(y))²+(G _(x) −G_(y))²);

Sentiment_difference=(100*G _(x) +H _(x))−(100*G _(y) +H _(y));

Normalization_variable=Z;

Consider jump if Sentiment_difference*Z>UID_Distance:

Save Sentiment_difference*Z−UID_Distance;

-   -   (4) Evaluate all nodes surrounding the current node and find the        closest node with the largest positive sentiment delta. If        sentiment from the possible jump is greater than the predicted        sentiment (can be further constrained by limiting the actual        UID_Distance considered by the algorithm), the jumpable node is        used instead of the predicted node as the next move.

In this algorithm, x is the current node and y is the possible futurenode, whether predicted or jumpable. G is multiplied by 100 to enable0-9900. G is summed with H to enable a value of 1-9999. This is areduction to practice of an embodiment, but many other embodiments arepossible without deviating from the scope of the invention. Indeed, thenumber is not limited to 1-9999, nor are the total number of dimensionsfixed at 6 dimensions. Fewer or more dimensions may be used. The numberof dimensions should be sufficient to adequately describe the dataset,and thus is typically dependent on the complexity of the dataset.

FIG. 9 illustrates a computing system 900 configured to map disallowedstates and enable access to those states under certain conditionsthrough a search algorithm, according to an embodiment of the presentinvention. System 900 includes a bus 905 or other communicationmechanism for communicating information, and processor(s) 910 coupled tobus 905 for processing information. Processor(s) 910 may be any type ofgeneral or specific purpose processor, including a central processingunit (CPU) or application specific integrated circuit (ASIC).Processor(s) 910 may also have multiple processing cores, and at leastsome of the cores may be configured for specific functions. System 900further includes a memory 915 for storing information and instructionsto be executed by processor(s) 910. Memory 915 can be comprised of anycombination of random access memory (RAM), read only memory (ROM), flashmemory, cache, static storage such as a magnetic or optical disk, or anyother types of non-transitory computer-readable media or combinationsthereof. Additionally, system 900 includes a communication device 920,such as a transceiver, to wirelessly provide access to a communicationsnetwork.

Non-transitory computer-readable media may be any available media thatcan be accessed by processor(s) 910 and may include both volatile andnon-volatile media, removable and non-removable media, and communicationmedia. Communication media may include computer-readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media.

Processor(s) 910 are further coupled via bus 905 to a display 925, suchas a Liquid Crystal Display (LCD), for displaying information to a user.A keyboard 930 and a cursor control device 935, such as a computermouse, are further coupled to bus 905 to enable a user to interface withsystem 900. However, in certain embodiments such as those for mobilecomputing implementations, a physical keyboard and mouse may not bepresent, and the user may interact with the device solely throughdisplay 925 and/or a touchpad (not shown). Any type and combination ofinput devices may be used as a matter of design choice.

In one embodiment, memory 915 stores software modules that providefunctionality when executed by processor(s) 910. The modules include anoperating system 940 for system 900. The modules further include a statemapping and modification module 945 that is configured to configured toperform the various disallowed state mapping and access enablementprocesses discussed herein. System 900 may include one or moreadditional functional modules 950 that include additional functionality.

One skilled in the art will appreciate that a “system” could be embodiedas a personal computer, a server, a console, a personal digitalassistant (PDA), a cell phone, a tablet computing device, or any othersuitable computing device, or combination of devices. Presenting theabove-described functions as being performed by a “system” is notintended to limit the scope of the present invention in any way, but isintended to provide one example of many embodiments of the presentinvention. Indeed, methods, systems and apparatuses disclosed herein maybe implemented in localized and distributed forms consistent withcomputing technology, including cloud computing systems.

It should be noted that some of the system features described in thisspecification have been presented as modules, in order to moreparticularly emphasize their implementation independence. For example, amodule may be implemented as a hardware circuit comprising custom verylarge-scale integration (VLSI) circuits or gate arrays, off-the-shelfsemiconductors such as logic chips, transistors, or other discretecomponents. A module may also be implemented in programmable hardwaredevices such as field programmable gate arrays, programmable arraylogic, programmable logic devices, graphics processing units, or thelike.

A module may also be at least partially implemented in software forexecution by various types of processors. An identified unit ofexecutable code may, for instance, comprise one or more physical orlogical blocks of computer instructions that may, for instance, beorganized as an object, procedure, or function. Nevertheless, theexecutables of an identified module need not be physically locatedtogether, but may comprise disparate instructions stored in differentlocations which, when joined logically together, comprise the module andachieve the stated purpose for the module. Further, modules may bestored on a computer-readable medium, which may be, for instance, a harddisk drive, flash device, RAM, tape, or any other such medium used tostore data.

Indeed, a module of executable code could be a single instruction, ormany instructions, and may even be distributed over several differentcode segments, among different programs, and across several memorydevices. Similarly, operational data may be identified and illustratedherein within modules, and may be embodied in any suitable form andorganized within any suitable type of data structure. The operationaldata may be collected as a single data set, or may be distributed overdifferent locations including over different storage devices, and mayexist, at least partially, merely as electronic signals on a system ornetwork.

FIG. 10 is a flowchart 1000 illustrating a process for providing a pathwith a better solution, according to an embodiment of the presentinvention. The process begins with mapping trade space versus time for aplurality of scenarios at 1010. Each pathway in the plurality ofscenarios includes a plurality of nodes. Space and time are randomlyseeded at 1020 to select an initial possible starting point. Pathwaysleading more desirable outcomes are positively weighted at 1030.

While following a pathway from the initial starting point, a search isperformed for nearby nodes from otherwise unreachable outcomes thatprovide a better solution at 1040. In some embodiments, the searchingfor the nearby nodes from otherwise unreachable outcomes that provide abetter solution may include employing a random search for otherwiseunreachable nodes that are proximate within a predetermined distance inthe trade space to a current node of the pathway, but provide a bettersolution. When a node is found that is proximate within thepredetermined distance and is associated with a pathway that provides abetter solution at 1050, the system jumps to the proximate node andfollows the path to the better solution at 1060. The path with thebetter solution is then output at 1070.

In some embodiments, trade space versus time is mapped for allscenarios. In certain embodiments, the mapping of trade space versustime includes determining unique IDs (UIDs) for each node in the map.The map may be based on image data, and the UIDs may each include sixnumbers that are derived from PCA of three principal components of a 2Dwavelet transform of a respective image and three principal componentsof a DCT of the respective image. The six numbers may be between 1-100in some embodiments. The UIDS include numbers representing sentiment incertain embodiments. In some embodiments, two numbers between 1 and 99are used to represent sentiment, with multiplier between 0 and 100 on asecond one of the two sentiment numbers such that values between 1 and9999 are possible.

In some embodiments, two numbers of the UIDs are used to facilitatepattern lock and the pattern lock is utilized for cases where states areidentical to enable repeating states to occur and still retain asequence. In certain embodiments, upon choosing a next possible step,the system evaluates prior experiences from the pattern lock, chooses abest experienced outcome, searches adjacent UIDs that have beenpreviously experienced or simulated, and evaluates whether a potentialimprovement of outcome warrants choosing a new path that has not beenexperienced. In some embodiments, the system jumps to a most desirablepath of a plurality of paths with success outcomes, or jumps from asuccess path to a more desirable success path, when a node of the moredesirable or most desirable success path is proximate within thepredetermined distance to the current node. In certain embodiments, thepathways include positive, neutral, or negative sentiments. The pathwayswith neutral sentiments may become more positive or more negative astime advances. In some embodiments, the paths with better solutions aredetermined for self-driving cars, image recognition, trade negotiations,music, or any combination thereof.

FIG. 11 is a flowchart 1100 illustrating a process for generating a new,more favorable solution path, according to an embodiment of the presentinvention. The process begins with performing PCA on raw data at 1110 togenerate relational sequences in trade space. The trade space representsrelative relationships in the raw data. A plurality of scenarios areseeded with initial conditions at 1120. The initial conditions areseeded with real world data. A solution space is mapped based on thegenerated relational sequences at 1130.

Each relational sequence is saved into a positive, negative, or neutralbin based on its outcome at 1140. In some embodiments, the neutral binis a default for observations of natural phenomena in order to simulatenatural phenomena that cannot be changed. In certain embodiments, afrequency with which a path including a given node is chosen iscorrelated to a number of times that the node has appeared during pastsimulations. In some embodiments, when a negative note is inevitable onall paths, a chance that a jump is attempted is increased.

Random states are initialized to explore more preferable solutions at1150. The plurality of scenarios are replayed at 1160 to determine pinchpoints where path structures are proximate to one another and morefavorable outcomes can be jumped to. A new solution path is thengenerated at 1170 based on one or more jumps to more favorable outcomes.

In some embodiments, the PCA results in a UID including six numbers,with three of the six numbers resulting from principal components of a2D wavelet transform of a respective image and another three of the sixnumbers resulting from principal components of a DCT of the respectiveimage. In certain embodiments, the six numbers are between 1 and 100. Insome embodiments, the UID further includes numbers representingsentiment.

In certain embodiments, the UID includes two additional numbers that areused to facilitate pattern lock, and the pattern lock is utilized forcases where states are identical to enable repeating states to occur andstill retain a sequence. In some embodiments, upon choosing a nextpossible step, the system evaluates prior experiences from the patternlock, chooses a best experienced outcome, searches adjacent UIDs thathave been previously experienced or simulated, and evaluates whether apotential improvement of outcome warrants choosing a new path that hasnot been experienced. In some embodiments, the UID includes anadditional two numbers between 1 and 99 that are used to representsentiment, with a multiplier between 0 and 100 on a second one of thetwo sentiment numbers such that values between 1 and 9999 are possible.

FIG. 12 is a flowchart 1200 illustrating a process for providing a pathwith a better solution, according to an embodiment of the presentinvention. The process begins with reviewing all nodes in a pattern at1210 to determine a most likely node for a jump. Next, the systemevaluates whether the jump to the most likely node is desirable at 1220using a sentiment difference between the most likely node for the jumpand at least one node that Is not previously known to be a valid state,but is still proximate to the most likely node for the jump. From allpredicted moves, a largest sentiment difference is chosen at 1230.

The system evaluates all jumpable nodes within a predetermined proximitythat are not predicted by pattern recall at 1240. In some embodiments,the predetermined proximity is determined by a combination of a UIDdistance, a sentiment difference, and a normalization variable, where ajump is considered if the sentiment difference times the normalizationvariable is greater than the UID distance. A jumpable node within thepredetermined proximity with a largest positive sentiment differencebetween itself and the most likely node for the jump is selected at1250. A path with the jumpable node is then output as a path with abetter solution at 1260.

The process steps performed in FIGS. 4 and 10-12 may be performed by acomputer program, encoding instructions for the processor to perform atleast the processes described in FIGS. 4 and 10-12, in accordance withembodiments of the present invention. The computer program may beembodied on a non-transitory computer-readable medium. Thecomputer-readable medium may be, but is not limited to, a hard diskdrive, a flash device, RAM, a tape, or any other such medium used tostore data. The computer program may include encoded instructions forcontrolling the processor to implement the process described in FIGS. 4and 10-12, which may also be stored on the computer-readable medium.

The computer program can be implemented in hardware, software, or ahybrid implementation. The computer program can be composed of modulesthat are in operative communication with one another, and which aredesigned to pass information or instructions to display. The computerprogram can be configured to operate on a general-purpose computer, oran ASIC.

It will be readily understood that the components of various embodimentsof the present invention, as generally described and illustrated in thefigures herein, may be arranged and designed in a wide variety ofdifferent configurations. Thus, the detailed description of theembodiments of the systems, apparatuses, methods, and computer programsof the present invention, as represented in the attached figures, is notintended to limit the scope of the invention as claimed, but is merelyrepresentative of selected embodiments of the invention.

The features, structures, or characteristics of the invention describedthroughout this specification may be combined in any suitable manner inone or more embodiments. For example, reference throughout thisspecification to “certain embodiments,” “some embodiments,” or similarlanguage means that a particular feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment of the present invention. Thus, appearances of the phrases“in certain embodiments,” “in some embodiment,” “in other embodiments,”or similar language throughout this specification do not necessarily allrefer to the same group of embodiments and the described features,structures, or characteristics may be combined in any suitable manner inone or more embodiments.

It should be noted that reference throughout this specification tofeatures, advantages, or similar language does not imply that all of thefeatures and advantages that may be realized with the present inventionshould be or are in any single embodiment of the invention. Rather,language referring to the features and advantages is understood to meanthat a specific feature, advantage, or characteristic described inconnection with an embodiment is included in at least one embodiment ofthe present invention. Thus, discussion of the features and advantages,and similar language, throughout this specification may, but do notnecessarily, refer to the same embodiment.

Furthermore, the described features, advantages, and characteristics ofthe invention may be combined in any suitable manner in one or moreembodiments. One skilled in the relevant art will recognize that theinvention can be practiced without one or more of the specific featuresor advantages of a particular embodiment. In other instances, additionalfeatures and advantages may be recognized in certain embodiments thatmay not be present in all embodiments of the invention.

One having ordinary skill in the art will readily understand that theinvention as discussed above may be practiced with steps in a differentorder, and/or with hardware elements in configurations which aredifferent than those which are disclosed. Therefore, although theinvention has been described based upon these preferred embodiments, itwould be apparent to those of skill in the art that certainmodifications, variations, and alternative constructions would beapparent, while remaining within the spirit and scope of the invention.In order to determine the metes and bounds of the invention, therefore,reference should be made to the appended claims.

1. A computer-implemented method, comprising: performing principalcomponent analysis (PCA) on raw data, by a computing system, to generaterelational sequences in trade space, wherein the trade space representsrelative relationships in the raw data; mapping a solution space basedon the generated relational sequences, by the computing system;replaying a plurality of scenarios, by the computing system, todetermine pinch points where path structures are proximate to oneanother and more favorable outcomes can be jumped to; and generating anew solution path, by the computing system, based on one or more jumpsto more favorable outcomes.
 2. The computer-implemented method of claim1, further comprising: seeding the plurality of scenarios with initialconditions, by the computing system, wherein the initial conditions areseeded with real world data; and initializing random states, by thecomputing system, to explore more preferable solutions.
 3. Thecomputer-implemented method of claim 1, further comprising: saving eachrelational sequence into a positive, negative, or neutral bin based onits outcome, by the computing system.
 4. The computer-implemented methodof claim 3, wherein the neutral bin is a default for observations ofnatural phenomena in order to simulate natural phenomena that cannot bechanged.
 5. The computer-implemented method of claim 1, wherein afrequency with which a path including a given node is chosen iscorrelated to a number of times that the node has appeared during pastsimulations.
 6. The computer-implemented method of claim 1, wherein whena negative note is inevitable on all paths, a chance that a jump isattempted is increased.
 7. The computer-implemented method of claim 1,wherein the PCA results in a UID comprising six numbers, with three ofthe six numbers resulting from principal components of a 2D wavelettransform of a respective image and another three of the six numbersresulting from principal components of a discrete cosine transform (DCT)of the respective image.
 8. The computer-implemented method of claim 7,wherein the six numbers are between 1 and
 100. 9. Thecomputer-implemented method of claim 7, wherein the UID furthercomprises numbers representing sentiment.
 10. The computer-implementedmethod of claim 7, wherein the UID comprises two additional numbers thatare used to facilitate pattern lock, and the pattern lock is utilizedfor cases where states are identical to enable repeating states to occurand still retain a sequence.
 11. The computer-implemented method ofclaim 10, wherein upon choosing a next possible step, the method furthercomprises: evaluating prior experiences from the pattern lock, by thecomputing system; choosing a best experienced outcome, by the computingsystem; searching adjacent UIDs that have been previously experienced orsimulated, by the computing system; and evaluating, by the computingsystem, whether a potential improvement of outcome warrants choosing anew path that has not been experienced.
 12. The computer-implementedmethod of claim 7, wherein the UID comprises an additional two numbersbetween 1 and 99 that are used to represent sentiment, with a multiplierbetween 0 and 100 on a second one of the two sentiment numbers such thatvalues between 1 and 9999 are possible.
 13. The computer-implementedmethod of claim 1, wherein the PCA results in a UID comprising numbersderived from a dimensional reduction technique that provides relationalmeaning between the UID and other UIDs.
 14. A computer-implementedmethod, comprising: generating relational sequences in trade space, by acomputing system, wherein the trade space represents relativerelationships in the raw data; mapping a solution space based on thegenerated relational sequences, by the computing system; replaying aplurality of scenarios, by the computing system, to determine pinchpoints where path structures are proximate to one another and morefavorable outcomes can be jumped to; and generating a new solution path,by the computing system, based on one or more jumps to more favorableoutcomes.
 15. The computer-implemented method of claim 14, furthercomprising: seeding the plurality of scenarios with initial conditions,by the computing system, wherein the initial conditions are seeded withreal world data; and initializing random states, by the computingsystem, to explore more preferable solutions.
 16. Thecomputer-implemented method of claim 14, further comprising: saving eachrelational sequence into a positive, negative, or neutral bin based onits outcome, by the computing system, wherein the neutral bin is adefault for observations of natural phenomena in order to simulatenatural phenomena that cannot be changed.
 17. The computer-implementedmethod of claim 14, wherein a frequency with which a path including agiven node is chosen is correlated to a number of times that the nodehas appeared during past simulations.
 18. The computer-implementedmethod of claim 14, wherein when a negative note is inevitable on allpaths, a chance that a jump is attempted is increased.
 19. Thecomputer-implemented method of claim 14, wherein the relationalsequences in trade space are generated by performing principal componentanalysis (PCA) on raw data, and the PCA results in a UID comprisingnumbers derived from a dimensional reduction technique that providesrelational meaning between the UID and other UIDs.
 20. A computerprogram embodied on a non-transitory computer-readable medium, thecomputer program configured to cause at least one processor to: generaterelational sequences in trade space by performing principal componentanalysis (PCA) on raw data, wherein the trade space represents relativerelationships in the raw data; map a solution space based on thegenerated relational sequences; replay a plurality of scenarios todetermine pinch points where path structures are proximate to oneanother and more favorable outcomes can be jumped to; and generate a newsolution path, by the computing system, based on one or more jumps tomore favorable outcomes, wherein the PCA results in a UID comprisingnumbers derived from a dimensional reduction technique that providesrelational meaning between the UID and other UIDs.